This disclosure relates generally to input devices in a data processing system and more specifically to calibrating one or more haptic devices with a multi-touch display in the data processing system.
The term “haptics” refers to the sense of touch. Haptic technology involves human computer interaction devices that interface using a sense of touch applying forces, vibrations, and/or motions to the user. Multi-touch displays support sensing of multiple interaction points on a display surface. Example multi-touch systems typically include smart devices such as phones with touch sensitive screens and kiosks with touch screens. Multi-touch sensing extends beyond the detection of human fingertips and can include detection of a variety of physical objects and visual markers, depending on the sensing technology adopted.
The combination of one or more haptic devices with a visual display is known as a “hapto-visual system”. Such systems may be collocated, where the haptic interaction and visual display volumes overlap. Several hapto-visual systems have been developed since the late 1960s. Most hapto-visual systems comprise a liquid crystal display (LCD) or cathode ray tube (CRT) desktop monitor reflected in a half-silvered mirror with a haptic device mounted behind the mirror. The user observes a stereoscopic image projected by the monitor by looking at the mirror, such that the virtual object represented by the image appears to be located behind the mirror, and holds the haptic device. The combined effect is one of “hands on” interaction with a virtual object or environment, in which the user can see and touch virtual objects as if the virtual objects were physical objects. Typical large-scale systems however use a different design where a large haptic device is situated in front of a large display surface.
The hapto-visual systems described above incorporate a robotic device for force-feedback. Alternative haptic devices have been integrated with display devices, including: tactile displays, such as technology that provides either limited display deformation or vibration feedback; physical widgets, such as silicone illuminated active peripheral (SLAP) widgets that use passive resistance to simulate control widgets on the display surface; and the use of magnetic induction to move objects on a display surface, such as provided by a Proactive Desk.
Personal hapto-visual systems typically limit collaboration between physically adjacent colleagues. Only one user can interact with such a system at a given moment. Personal hapto-visual systems also limit visual context due to their use of small-scale display devices. Increasing the number of haptic devices in a personal hapto-visual system, for example to support bimanual interaction, is difficult due to overlapping physical work volumes and a high probability of collision between haptic devices. Large-scale hapto-visual systems limit the work volume accessible by the haptic device to a fixed subset of the visual display surface and additionally obscure substantial portions of the visual display with parts of the haptic device. Large-scale hapto-visual systems also pose safety risks due to the close proximity of a large robotic (haptic) device to the body and head of a user while the attention of the user is focused on the visual display.
Tactile displays and active peripherals limit the haptic working volume to the display plane. A user only feels haptic feedback while the fingers of the user are in contact with the display surface, which is a severe restriction in stereoscopically projected visual virtual environments.
In the field of haptic devices, several current solutions are available including a two dimensional hapto-visual system using an electro-magnetic device or a resistive ballpoint in a stylus (pen) for obtaining force-feedback. Interaction with multiple styluses is described. The two dimensional hapto-visual system is essentially an “active peripheral”, however, with limitations.
Another example of a solution may be viewed as a trivial extension of a vibro-tactile display to accommodate multi-touch sensing. A similar variation of the previously stated vibro-tactile display system provides an emphasis on non-visual feedback in portable devices, for example, by providing a capability to locate graphic user interface widgets without examining the display.
In another example of a current solution, a variant of a tactile display is presented in which the surface of the display deforms either in response to a touch of the user or to emphasize two dimensional graphic user interface (GUI) elements such as active button widgets. The example solution has similar limitations as other tactile displays currently available.
In another example of current solutions, a combination of displays (some touch-sensitive, some not) and physical buttons (such as those on a game controller) provides a capability in which the device as a whole may vibrate or the physical buttons may provide force-feedback. The motion or feedback may be, for example, refusing to depress, in response to the combination of something displayed on one of the displays and a physical button or touch-screen press from a user.
In another example a haptic stylus provides a variety of vibration effects. For example, a vibrating haptic stylus is similar to tactile display systems except the actuator providing haptic feedback is mounted in the stylus rather than in the display. A further variation on tactile multi-touch displays, as previously described, includes a tactile element attached to the fingers of a user rather than the stylus or the display as stated previously.
Existing hapto-visual systems limit the scale of collocated hapto-visual interaction, impede collaboration between multiple simultaneous users and, in larger systems, risk safety and increase cost through placement of large robotic devices in close proximity to the head of a user.